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DETAILED ACTION 
Drawings 

1. The drawings figure 4 are objected to as failing to comply with 37 
CFR 1.84(p)(4) because reference character "31" has been used to designate both 
ambient image and occupant. Corrected drawing sheets in compliance with 37 CFR 
1.121(d) are required in reply to the Office action to avoid abandonment of the 
application. Any amended replacement drawing sheet should include all of the figures 
appearing on the immediate prior version of the sheet, even if only one figure is being 
amended. The replacement sheet(s) should be labeled "Replacement Sheet" in the 
page header (as per 37 CFR 1.84(c)) so as not to obstruct any portion of the drawing 
figures. If the changes are not accepted by the examiner, the applicant will be notified 
and informed of any required corrective action in the next Office action. The objection to 
the drawings will not be held in abeyance. 

Specification 

2. The disclosure is objected to because of the following informalities: 

(a) Paragraph (0061) refers to figure 10 and points to a centroid labeled as 84 
(line 3) and 54 (line 5) whereas figure 10 shows the centroid labeled as 82. 
Specification in paragraph 0061 is not consistent to figure 10. 

(b) Paragraph (0065) refers to image thresholding subsystem 100 which 
comprises a set image threshold module (labeled as 40 -line 2) and a perform image 
thresholding module (labeled as 42 - line 2). Figure 1 1 comprises image thresholding 
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subsystem 100 which comprises a set image threshold module (labeled as 104) and a 
perform image thresholding module (labeled as 106). Specification in paragraph 0065 is 
not consistent to figure 1 1 . 

(c) Paragraph (0077), line 12, starts with word "They system 16 ". Examiner 

suggests to change the word "they" to "the". 

(d) Paragraph (0081), line 7, has word "replaced" in "such as luminosity in the 
preferred embodiment, can replaced". Examiner suggests to change the word 
"replaced" to "replace". 

Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-4, 6-13, and 20-33 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Baxes, Book Publication, 1994, "Digital image processing: principles 
and application", further in view of Ishikawa et al., U.S. Patent No. 4,625,329. 

Regarding claim 1, Baxes teaches that the image segmentation process 
consists of three stages (i) image preprocessing, where the image is visually improved 
to make it as free of zero-information-carrying clutter as possible, (ii) initial object 
discrimination, where objects are grossly separated into groups with similar attributes, 
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and (iii) object boundary cleanup which includes image morphological processing, 
where object boundaries are reduced to single-pixel widths and removes noise clutter 
and other artifacts from the image (page 124, para. 2, figure 5.1). Baxes further 
discloses that an image exists as a large array of pixels with associated brightness 
(characteristic) (page 123, para. 2, line 1). It is clear from this disclosure by Baxes that 
image to be segmented consists of a plurality of pixels and each pixel inherently has a 
value (initial value) associated with it to measure the characteristics associated which 
conforms to the limitation "receiving an ambient image represented by a plurality of 
pixels and plurality of initial pixel values, wherein each said pixel has at least one 
said initial pixel value". 

Baxes teaches the second stage (initial object discrimination) of the image 
segmentation process, which uses binary contrast enhancement operation to isolate 
objects in the image, by highlighting similar objects with a common brightness (page 
126, para. 3). Baxes further discloses binary contrast enhancement consists of 
comparing each pixel (initial) value to the threshold value to identify whether pixel 
belongs to darker (background) area or the brighter (foreground or object) area where 
darker and brighter areas are two pixel categories (page 330) which conforms to the 
limitation "identifying one or more pixels as belonging to one of a plurality of pixel 
categories on the basis of one or more initial pixel values associated with the 
pixels being identified". 

Baxes further teaches binary contrast enhancement process consists the 
comparing of pixel brightness (initial) value to the brightness threshold value where if 
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the pixel brightness is darker than the threshold, the resulting pixel brightness is set to 0 
(first revised pixel value) and if the pixel brightness is brighter than the threshold, the 
resulting brightness is set to 255 (first revised pixel value) (page 330, para 1). It is clear 
from this disclosure that each pixel in the brighter area will have a value of 255 and 
each pixel in the darker area will have a value of 0 which conforms to the limitation 
"establishing a first revised pixel value for one or more pixels, wherein each pixel 
in the same pixel category has the same first revised pixel value". 

Baxes teaches the third stage (object boundary cleanup) of the image 
segmentation process, which uses a binary morphological processes (operations) such 
as erosion and dilation (opening and closing), to operate upon binary images, such as 
those created using a binary contrast enhancement operation (page127, para. 2; page 
128, para. 4 and 5). Baxes further teaches of comparing every pixel in the image with its 
8 neighbors to produce a resulting output pixel value (second revised pixel value) (page 

128, last para.). Baxes further teaches "In case where all nine input pixels are identical 
to their respective mask values, the resulting pixel value is set to a predefined logical 
value (either 0 or 1). Where one or more input pixels do not match their respective mask 
values, the resulting value is set to the opposite state (revised second pixel value) (page 

129, para.1) which conforms to the limitation "setting a second revised pixel value 
for one or more of said pixels on the basis of one or more first revised pixel 
values that are associated with one or more pixels in the vicinity of the pixel 
being set". 
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The whole disclosure by Baxes, discussed above would conform to the limitation 
"deriving a segmented image of the occupant (object) from said first revised pixel 
value and said second revised pixel value". 

Baxes further discloses "the mission of image analysis operations (such as 
image segmentation) is to understand an image by quantifying its elements. The 
elements of interest are generally objects in an image, such as cells, bolts, aircraft, or 
characters on a page of text, which are occupant of the image. Image analysis 
operations play a major role in automated machine vision and image interpretation 
applications of digital image processing" (page 123, para. 1). It is clear form above 
disclosure that image processing operations as disclosed by Baxes can be used in any 
image processing field where image segmentation is required to identify the object 
(occupant) in the image and this is further supported by Ishikawa. Ishikawa discloses of 
a similar image segmentation method and system, which includes an image sensor for 
generating sensor measurements to identify the occupant in the vehicle (figures 8, 10A 
and 10B; col. 1, lines 56-60; col. 2, lines 28-32 and lines 45-46; col.5, lines 15-50). 

Therefore, it would have been obvious for one of ordinary skill in the art at the 
time of the invention was made to use Baxes's teachings in the machine vision and 
image interpretation applications related to image processing such as disclosed in 
Ishikawa's invention. 

Claim 2 recites "an image segmentation method as in claim 1, wherein 
identifying one or more said pixels further includes generating an image threshold". 
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Baxes discloses determining (generating) an image threshold (page 77, para. 2, lines 3- 
5). 

Claim 3 recites "an image segmentation method as in claim 2, wherein 
identifying one or more said pixels further includes comparing said plurality of initial 
pixel values to said image threshold". Claim 3 has been similarly analyzed and rejected 
as per the rejection of claim 1 and 2. 

Claim 4 recites "an image segmentation method as in claim 2, wherein 
generating an image threshold further comprises analyzing the distribution of initial pixel 
values relating to a pixel characteristic". Baxer discloses determining the threshold by 
computing the average brightness in the area (page 77, lines 3-4; page 153, para. 3) 
and further teaches that brightness and color features (pixel characteristics) of an object 
can be extracted by examining every pixel within the object's boundaries and discloses 
that this can be done using a histogram, where histogram shows the brightness (pixel 
characteristic) distribution found in the object (page 153, para. 2 and 3). 

Claim 6 recites "an image segmentation method as in claim 4, wherein each 
pixel has only one initial pixel value and only one pixel characteristic". Baxes as 
discussed in the rejection of claim 4, uses a histogram where histogram provides the 
characteristic (brightness) associated with each pixel, It is clear from this that in 
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histogram graph (page 63, figure 3.19b) each pixel will have one value for each 
characteristic (sharpness) associated with it. 

Claim 7 recites "an image segmentation method as in claim 6, wherein 
luminosity is said pixel characteristic". Luminosity is the synonym of Brightness as the 
characteristic of the pixel. Claim 7 has been similarly analyzed and rejected as per 
claims 6, 4 and 1 . 

Claim 8 recites "an image segmentation method as in claim 3, wherein each said 
pixel has a pixel location with respect to the ambient image, and said pixel location 
determines which of a plurality of image thresholds are compared to said initial pixel 
value for said pixel in said pixel location". As disclosed in the rejection of claim 1, Baxes 
teaches that an image exists as a large array of pixels with associated brightness 
(characteristic) (page 123, para. 2, line 1). It is clear from this disclosure by Baxes that 
image is made of plurality of pixels and each pixel has to have a any pixel location to 
represent the image and the position of the pixel can further be determined using 
histogram which shows the distribution of image pixels. Baxes further discloses in the 
rejection of claim 1, that for image segmentation the initial pixel value of the pixel is 
compared to the image threshold generated. Baxes further teaches adaptive 
thresholding in which the appropriate threshold value may need to change throughout 
the image being processed depending on the qualities of the image in the area being 
processed, for example, "in areas where the image is darker, the threshold value 
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decrease and in areas where the image is lighter (brighter), the threshold value 
increases" (page 76, para. 1). It is clear from the above disclosure by Baxes, adaptive 
thresholding can be used to determine a plurality of image thresholds depending on the 
pixel location, to which the each said pixel can be compared to, which conforms to the 
limitation "said pixel location determines which of a plurality of image thresholds 
are compared to said initial pixel value for said pixel in said pixel location". 

Claim 9 recites "an image segmentation method as in claim 8, wherein a higher 
image threshold is applied in pixel locations where there is brighter lighting". Claim 9 
has been similarly analyzed and rejected as per claim 8. 

Claim 10 recites "an image segmentation method as in claim 1, wherein there 
are only two pixel categories". As disclosed by Baxes in the rejection of claim 1, The 
pixel is either identified as belonging to darker area or brighter area where darker and 
brighter areas are two pixel categories. Claim 10 has been similarly analyzed and 
rejected as per claim 1 . 

Claim 11 recites "an image segmentation method as claim 1, wherein setting the 
second revised pixel value includes a morphological heuristic". As disclosed by Baxes in 
rejection of claim 1, the third stage (object boundary cleanup) of the image 
segmentation process, which uses a binary morphological processes (heuristic) such as 
erosion and dilation, to operate upon binary images, such as those created using a 
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binary contrast enhancement operation (page127, para. 2; page 128, para. 4 and 5) and 
Baxes further teaches of comparing every pixel in the image with its 8 neighbors to 
produce a resulting output pixel value (second revised pixel value) (page 128, last 
para.). Claim 1 1 has been similarly analyzed and rejected as per claim 1. 

Claim 12 recites "an image segmentation method as in claim 11, wherein the 
morphological heuristic is a morphological erosion". Claim 12 has been similarly 
analyzed and rejected as per claim 1 1 and claim 1. 

Claim 13 recites "an image segmentation method as in claim 12, wherein the 
morphological heuristic is a morphological dilation". Claim 13 has been similarly 
analyzed and rejected as per claims 12, 11 and 1. 

Claim 20 recites an image segmentation system, which comprises of an image 
thresholding subsystem and a gap processing subsystem. As discussed in the rejection 
of claim 1, Baxes discloses image segmentation, which comprises of three stages 
where second stage provides the image thresholding and third stage uses dilation and 
erosion (closing and opening) for a gap processing (page 136, para. 5, lines 2-3). To 
perform these image segmentation operations, a system is obviously required. Ishikawa 
presents a image processing system for image segmentation as discussed in claim 1. 
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Claim 21 has been similarly analyzed and rejected as per claims 20, 1, 4, 6 and 

7. 

Claim 22 has been similarly analyzed and rejected as per claims 21, 20, 1, 4, 6, 

and 7. 

Claim 23 has been similarly analyzed and rejected as per claims 22 -> 20 and 5. 

Claim 24 has been similarly analyzed and rejected as per claims 23 -> 20 and 5. 

Claim 25 has been similarly analyzed and rejected as per claims 24 -> 20 and 8. 

Claim 26 recites two subsets of pixels similarly like two categories (darker area 
and brighter area which contain dark and white pixels) as in rejection of claim 1. Claim 
26 has been similarly analyzed and rejected as per claims 24 -> 20 and claim 1. 

Claim 27 has been similarly analyzed and rejected as per claims 26 -> 20 and 
claim 1. 

Claim 28 recites "an image segmentation system as in claim 27 wherein at least 
approximately half of said plurality of said pixels are set to said first binary value". 
Setting the plurality of said pixels to said first binary value depends on the size of the 
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object to be segmented and the on the area of the background, if the area of the object 
is half the image, then it is clear that almost half the pixels in the image will be set to 
binary value to be labeled to the object and rest to the background. 

Claim 29-31 have been similarly analyzed and rejected as claims 20, 1 and 11- 

13. 

Claim 32 recites "an image segmentation as in claim 29, said morphological 
heuristic comprising a vertically-based morphological heuristic". Baxes further discloses 
that morphological erosion and dilation operations can be performed in vertical and 
horizontal directions (page 136, para. 1, line 3). 

Claim 33 recites "an image segmentation as in claim 29, said morphological 
heuristic comprising a horizontally-based morphological heuristic". Claim 33 has been 
similarly analyzed and rejected as per claim 32. 

5. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Baxes, 
Book Publication, 1994, "Digital image processing: principles and application", further in 
view of Ishikawa et al., U.S. Patent No. 4,625,329 and further in view of Google article, 
Jan. 29, 2001, "Point operations", and further in view of Hentea, IEEE Publication, 1993, 
"Algorithm for automatic threshold determination for image segmentation". 
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Claim 5 recites "an image segmentation method as in claim wherein analyzing 
the distribution of initial pixel values further includes: recording aggregate initial pixel 

values into a histogram; translating the histogram into a cumulative distribution 

N f 

■ \ 

function". As disclosed in the rejection of claims 1 and 4, Baxes discloses the use of 
histogram to generate the image threshold by recording the initial pixel values of the 

image in the histogram. Baxes does not teach of translating the histogram into a 

> V 

cumulative distribution function and calculating an image threshold based on the 
predetermined percentage of initial pixekvalues falling below the image threshold and 
neither does Ishikawa teaches the same. It is well known in the art of image processing 
that thresholding in its various versions (local, global, and dynamic), is the most 
commonly used method for image segmentation and global thresholding produces 
satisfactory results if the input images are corrected for uneven illuminant (shading 
correction). Translating vthe histogram into a cumulative distribution function is nothing 
but histogram equalization and histogram equalization is well known in the art of image 
processing and is further supported by Google article "Point operations". This article on 
page 2 of 4, discloses histogram equalization! by obtaining the cumulative probability 
distribution of r (where r is an input value). Therefore, it would have been obvious for 
one of ordinary skill in the art at the time of invention was made, to use the teachings of 
"Point operations" of obtaining the cumulative probability distribution of the histogram in 
the combined invention of Baxes and Ishikawa. One would have been motivated to use 
the teachings of "Point operations" of obtaining the cumulative probability distribution of 
the histogram in the combined invention of Baxes and Ishikawa because both the all the 
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- reference are directed to image processing field and "Point operations" performs 
i; (histogram equalization by obtaining the cumulative probability distribution of the 
histogram to distribute the gray levels of the image evenly over the maximum 
possible range (page 2 of 4). 

Claim 5 further recites the limitation "calculating an image threshold based on a 
n predetermined percentage of initial pixel values falling below the image threshold". 
Baxes does teach of determining image threshold based on the calculation performed 
on the initial pixel values but calculating an image threshold based on the 
predetermined percentage of initial pixel values falling below the image threshold and 
neither does Ishikawa and "Point operations" teaches the same. However, Hentea 
^discloses an algorithm for automatiathreshold selection for the detection of objects that 
j> are darker than the background (page 535, right col., para. 2). Hentea further provides 
the teachings where a cumulative histogram is calculated from the pixel values and then 
the image threshold is automatically selected as the gray level for which a given 
percentage of the blocks (or plurality of pixel values) have minimum values lower than it 
(image threshold) (page 535, right col., para. 4). 

Therefore, it would have been obvious for one of ordinary skill in the artat the 
time of the invention was made to use the teachings of Hentea in the combined 
invention of Baxes, Ishikawa and "Point operations". One would have been motivated to 
use the teachings of Hentea in the combined invention of Baxes, Ishikawa and "Point 
c, operations" because Hentea belongs, to the same art of endeavor (image segmentation) 
t» as- are Baxes and Ishikawa and Hentea further provides the teachings for the automatic 
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selection of the threshold for global thresholding which is very robust and performs well 
for images belonging to a large class and is also very efficient from a computation point 
of view, and is very flexible (page 538, right col., para. 1). 

Allowable Subject Matter 

6. Claims 18 and 19 are allowed. 

The following is an examiner's statement of reasons of allowance: 

Both the instant invention and the closest prior art Baxes, Book 
Publication, 1994, "Digital image processing: principles and application" are directed to 
a method of image segmentation to identify objects (occupants) in the images. The 
instant invention further recites the limitation "determining a third revised pixel value 
from said second revised pixel value or said first revised pixel value with a momentum- 
based heuristic" in claim 18 whereas Baxes does not teach use of momentum-based 
operations or heuristic to determine a third revised pixel value from said second revised 
pixel value or said first revised pixel value. Therefore claim 18 is allowed. Claim 19 is 
dependent on claim 18 and therefore is allowed. 

7. Claims 14-17 are objected to as being dependent upon a rejected base claim 1, 
but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 

The following is an examiner's statement of reasons of allowance: 
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The instant invention is directed to a method of image segmentation to identify 
the object (occupant) in the image. The key difference between the instant invention 
and the closest prior art (Baxes, Book Publication, 1994, "Digital image processing: 
principles and application") is the fact that the instant invention uses morphological 
heuristic, momentum-based heuristic and gravity based heuristic for image 
segmentation process whereas Baxes only use a morphological heuristic for image 
segmentation. The limitations "an image segmentation method as in claim 1, wherein 
setting the second revised pixel value includes a momentum-based heuristic" as recited 
in claim 14 and "an image segmentation method as in claim 1, wherein setting the 
second revised pixel value includes a gravity-based heuristic" as recited in claim 17 are 
not disclosed or suggested by prior art of record. Therefore, claims 14 and 17 are 
allowed. Claims 15 and 16 are dependent on claim 14, therefore they meet the 
conditions of allowance as applied to claim 14. 

8. Claims 34-38 are objected to as being dependent upon a rejected base claim 20, 
but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 

The following is an examiner's statement of reasons of allowance: 
The same reasons of allowance are applicable to claim 34 and 37 as applied to 
the allowance of claims 14-17. The limitations "An image segmentation system as in 
claim 20, said gap processing heuristic comprising a momentum-based heuristic and 
said subset of vicinity pixels comprising a subset of sequential pixels, and said gap 
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processing subsystem selectively setting said pixel value using said momentum-based 
heuristic and said pixel values relating to said subset of sequential pixels" as recited in 
claim 34 and "said gap filling heuristic comprising a gravity-based heuristic, a pixel 
region, and a region characteristic" as recited in claim 37, are not disclosed or 
suggested by prior art of record. Therefore, claims 34 and 37 are allowed. Claims 35 
and 36 are dependent on claim 34, therefore they meet the conditions of allowance as 
applied to claim 34. Similarly claim 38 is allowed, being dependent on claim 37. 

Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

• Lipton et al., U.S. Patent No. 6,625,310, discloses a method for 
segmenting video data into foreground and background portions utilizes 
statistical modeling of the pixels and further uses morphological erosion 
and dilation. 

• Young et al., U.S. Patent No. 6,249,590, discloses a method for 
automatically locating image pattern in digital images. 

• Owechko et al., U.S. Patent No. 6,801,662, discloses a vision-based 
system for automatically detecting the type of object within a specified 
area, such as the type of occupant within a vehicle. 
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• Vaidyanathan et al., U.S. Patent No. 5,982,944, discloses image analysis 
methods and systems, which are used for identifying objects in the 
background by defining a data space, such as histogram or a color space. 

• Luo, U.S. Patent No. 6,697,502, discloses a image processing method for 
detecting human figures in a digital image. 

• Kamei et al., U.S. Patent No. 5,528,698, discloses an automotive 
occupant-sensing device. 

• Kamgur-Parsi, U.S. Patent No. 5,923,776, discloses object extraction in 
images. 

• Florent, U.S. Patent No. 5,832,111, disclose a method and device for 
binary segmentation of a signal representing a digital image with a 
threshold obtained using accumulated (cumulative) histogram. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Manav Seth whose telephone number is (571) 272- 
7456. The examiner can normally be reached on Monday to Friday from 8:30 am to 
5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
trainer, Joseph Mancuso, can be reached on (571) 272-7695. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 




